Using documents

An agent can use one or multiple documents, and the same documents can be shared across multiple agents.

To use a document:

  1. In the Navigation menu pane, expand AI agents, and then click Agents.
  2. Create a new AI Agent, or select and existing one, and click Edit.
  3. In the Documents tab,
    1. From the Document mode drop-down list, select one of the supported modes. For details, see Supported modes.
    2. Add the relevant documents.

Supported modes

Supported modes for working with documents.

Note: These modes work with the document as free text — retrieved by semantic search or injected whole. For tabular documents (Excel / CSV) where you want to look a record up by one or more specific columns — for example by name, or by phone number and city — a structured lookup is usually a better fit.

Semantic search (for every query)

Implements classic Retrieval-Augmented Generation (RAG) pipeline:

This mode is mostly applicable to simple Q&A agents.

Note: Avoid this mode when the agent also needs to call tools. Because relevant extracts are appended to every user query, they may “distract” the LLM from following the conversation context and from deciding when to invoke its tools. For a tool-using agent, prefer Semantic search (agentic, via doc_search tool), which retrieves extracts only when the LLM decides it needs them.

Semantic search (agentic, via doc_search tool)

Implements agentic RAG pipeline, where LLM decides on its own when it needs document extracts and gets them via the doc_search tool, implicitly added to the agent. The system prompt lists the name and description of every document the tool searches, so the LLM knows what it can find there.

This mode co-exists much better with other tools. It may also slightly reduce agent’s costs, because the LLM may sometimes skip extracts generation, for example, for initial “Hello” message from user.

Note: This is the recommended mode for most cases where semantic search is needed. It suits both pure Q&A and tool-using agents, and retrieves document extracts only when the LLM actually needs them.

An advanced mode may be activated by setting the doc_search mode parameter to “Advanced”. In this mode, the LLM needs to specify name of the document in which it searches for relevant extracts, rather than perform the search in all attached documents.

Full content (in prompt)

Includes complete content of attached documents in the prompt. This mode is suitable for relatively short documents where complete document content is required for proper agent behavior.

For example, consider an agent that recommends a room in the hotel suitable for customer needs. That agent has access to a document that describes all types of rooms in the hotel. If you rely on RAG, your agent may get partial document extracts, where some types of rooms are missing, and thus fail to do its task. Including complete document content in prompt will mitigate this problem.

Full content (agentic, via doc_get tool)

Implements agentic pipeline, where the LLM can request content from specific document via the doc_get tool, implicitly added to the agent.

This mode allows you to attach multiple documents to the agent, and let agent access the specific document content when it’s needed.

An advanced mode may be activated by setting the doc_get mode parameter to “Advanced”. In this mode, the LLM can learn table of contents of specific document (deduced from its headers) via the doc_toc tool, implicitly added to the agent and then request specific section of the document instead of the full document content.

Per-document mode

By default the Document mode drop-down applies the same mode to all of the agent's documents. The documents_mode advanced configuration parameter overrides the mode per document, so a single agent can, for example, inline a short policy document into the prompt, expose a large manual via the doc_get tool, and keep a knowledge base on semantic doc_search – all at once.

Parameter Type Description
documents_mode dict[str, str] Per-document mode override; key = document name, value = mode. Documents not listed inherit the agent-level Document mode.

The mode value is one of:

Value Mode
rag Semantic search (for every query)
doc_search Semantic search (agentic, via doc_search tool)
prompt Full content (in prompt)
doc_get Full content (agentic, via doc_get tool)

Behavior:

Example

Inline a short policy, expose a manual via doc_get, and search a knowledge base – while any other attached document keeps the agent-level Document mode:

{
    "documents_mode": {
        "Refund policy": "prompt",
        "Product manual": "doc_get",
        "Knowledge base": "doc_search"
    }
}

Document tool results in conversation context

By default, only the latest response of the doc_search tool is kept in the conversation context. When the LLM calls doc_search again, the previous response is redacted, so that document extracts don't accumulate and increase token consumption. The LLM still sees that it called the tool, so it can call it again if it needs the earlier extracts once more.

By default, all responses of the doc_get and doc_toc tools are kept in the conversation context.

You may change this behavior per tool via the results_in_context setting of the doc_tools advanced configuration parameter. It keeps the last N responses of the tool in the conversation context and redacts the rest:

Value Meaning
N (1, 2, …) The last N responses are kept; older ones are redacted.
0 No response is kept – each one is redacted as soon as the LLM has answered from it.
-1 All responses are kept.

You may also customize the default message that is used in place of a redacted response via the redact_msg setting – for example, to tell the LLM how to get the content back.

Note: On agents using speech-to-speech models (e.g. gpt-realtime), document tool responses are not redacted.

Example

Keep the last two doc_search responses, and redact the older ones:

{
    "doc_tools": {
        "doc_search": {
            "results_in_context": 2
        }
    }
}

Controlling amount of consumed documents data

“Semantic search (for every query)” and “Semantic search (agentic, via doc_search tool)” modes by default use 5 document extracts semantically close to user query. You may customize this number via rag_chunks advanced configuration parameter to optimize retrieval performance based on your specific use case. Increasing the number of chunks provides more comprehensive context for complex queries but may lead to higher token usage and processing costs, while reducing the number creates more focused responses with lower computational overhead but potentially less complete information coverage.

{
    "rag_chunks": 10
}

In “Full content (in prompt)” and “Full content (agentic, via doc_get tool)” modes, you may use doc_content_len advanced configuration parameter to limit the total length of the consumed document content. This allows you to manage token usage and processing costs while also improving LLM response times.

{
    "doc_content_len": 50000
}

Filtering documents by condition

The document_conditions advanced configuration parameter restricts which of the agent's documents are available, based on conditions. This lets you expose a document only in specific situations – for example, offer a “Premium plan” document only to premium callers.

Parameter Type Description
document_conditions list[DocumentCondition] Conditions that gate the documents available to the agent.
DocumentCondition
Parameter Type Description
document str Name of the document the condition applies to.
condition str Condition that must evaluate to true for the document to be available. Written using the expression syntax and evaluated against the agent variables and conversation data – for example, caller == 123456.

Behavior:

When conditions are evaluated

Example

Expose the Premium plan document only to a specific caller:

{
    "document_conditions": [
        {
            "document": "Premium plan",
            "condition": "caller == 123456"
        }
    ]
}